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ABSTRACT 

NASA’s Aviation Safety Program, Synthetic Vision Systems Project is conducting research in advanced flight deck 
concepts, such as Synthetic/Enhanced Vision Systems (S/EVS), for commercial and business aircraft. An emerging 
thrust in this activity is the development of spatially-integrated, large field-of-regard information display systems. Head- 
worn or helmet-mounted display systems are being proposed as one method in which to meet this objective. System 
delays or latencies inherent to spatially-integrated, head-worn displays critically influence the display utility, usability, 
and acceptability. Research results from three different, yet similar technical areas - flight control, flight simulation, and 
virtual reality - are collectively assembled in this paper to create a global perspective of delay or latency effects in head- 
worn or helmet-mounted display systems. Consistent definitions and measurement techniques are proposed herein for 
universal application and latency requirements for Head -Worn Display S/EVS applications are drafted. Future research 
areas are defined. 
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1. INTRODUCTION 

The Synthetic Vision Systems (SVS) project, under NASA’s Aviation Safety Program (AvSP), is developing 
technologies with practical applications that will eliminate low visibility conditions as a causal factor to civil aircraft 
accidents 1 while replicating the operational benefits of clear day flight operations, regardless of the actual outside 
visibility condition. Research, development, test, and evaluation, including ground simulation and flight testing, have 
been conducted to mature SVS concepts (Figure 1), which provide a real-time, unobscured view of the out-the-window 
terrain and obstacle information by visually rendering an on-board terrain database, with integrity and reliability 
sufficient for primary flight reference. 2, 3 ’ 4 

1.1 Advanced Information Display Systems Using Synthetic/Enhanced Vision System 

The optimal fusion of Synthetic and Enhanced Vision Systems (S/EVS) technology is emerging as a cornerstone to the 
development of advanced flight deck information systems which can provide the flight crew with significantly improved 
spatial awareness, increased awareness of outside terrain and obstacle features, enhanced manual flight performance, and 
reduced pilot workload. The goal is to develop technologies which create an “error-immune” flight deck information 
system with potentially revolutionary improvements in safety and capacity for commercial flight operations, enabling 
safety and capacity in all-weather conditions superior to present, clear-day operations. A key component to these 
revolutionary benefits is advanced display media which can create spatially -integrated, large field-of-view, “unlimited” 
field-of-regard information displays for the pilot (Figure 1). 

Head-worn or helmet- mounted displays (HMDs) are one display media which may meet these information display 
requirements. HMDs are not new technology, particularly for military operations, but component miniaturization and 
maturation are progressing to the point where HMDs can be considered in commercial and business aircraft operations - 
i.e., the costs are reaching affordable levels and their use should be as unobtrusive to the pilot as wearing sunglasses. 

The head- worn display S/EVS application will optimally blend unlimited field-of-regard database (synthetic) and large 
field-of-view sensor (enhanced) imagery onto large field-of-view display media with various flight symbology and 
information to create an intuitive, spatially integrated cue environment for the pilot. The HMD will include a 
combination of aircraft-referenced, earth-referenced, and screen-referenced information and symbology 5 to enable safety 
and capacity superior to Visual Flight Rules operations today. Numerous technical challenges, in general, and for 
HMDs, in particular, have been identified as barriers which may inhibit full realization of these goals. 



1.2 Latency Requirements Development 


A critical determinant influencing the utility, usability, and acceptability of HMDs is the system latency for imagery and 
symbology drawn in either aircraft-referenced or Earth-referenced frames on large field-of-view displays - both key 
elements of the S/E VS application. 
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Figure 1: NASA Synthetic Vision System Concept And Potential HMD Application 

The topic of latency as it influences the development and use of HMD systems has been addressed from three different, 
yet similar technical areas - flight control, flight simulation, and virtual reality. For example, in flight control 
applications, time delay has caused aircraft accidents and incidents 6,7,8 as latency between a pilot’s cockpit control inputs 
and the aircraft response can dramatically degrade the pilot’s ability to control the aircraft. In flight simulation, time 
delay in simulating an aircraft response can negatively affect transfer-of-training and possibly induce simulator 
sickness. 9,10,11 In virtual reality, time delay can detract from the system’s usability and negatively impact the fidelity of 
the application or in the creation of “presence;” 12,13,14,15 i.e., the state of being part of the virtual world. At the union of 
these technologies, a global perspective of latency effects in head-worn or helmet-mounted display systems is possible. 

2. BACKGROUND 


2.1 Time Delay Sources 

A system, which reproduces the exact form of an input after a specific interval of time, is defined as a time delay. Time 
delays have also been referred to as “latency;” 16 others use the terminology of “lag.” 17,18 While each of these terms have 
been used in an attempt to be descriptive, confusion has more likely resulted because of differences in measurement 
techniques used to quantifying the effects for other users. The problem is that the delay of a system is created from a 
multitude of sources, most of which are not typically thought of as “pure” digital time delays but most of which add 
significant, if not, the majority of the latency. In this paper, the term “latency” is used to describe the technical problem 
yet it is not an engineering measurement. Engineering measurement of latency is addressed in the next section. 

Using Laplace notation, a delay element, shown in block diagram of Figure 2, is defined as: e' TS . Pure time delay (t) is a 
digital system effect. In a frequency domain analysis, time delay adds phase lag (4>) proportional to the input frequency 
(cd), as 4> = 57.29tco, where x is in seconds and 03 is in units of radians/sec. Examples of pure time delay sources are: 1) 
Computational delay; 2) Loitering delay; 3) Synchronization delay (e.g., “periodic delay” 19 ); and, 4) Sampling delay. 

The update rate of the system is a discrete system effect. The update rate does contribute to the latency, as noted in the 
digital delay source examples above. However, update rate, somewhat independent of the latency that it contributes, can 
be a powerful determinant in system usability because the update rate, if not fast enough, will cause noticeable discrete 
steps or “jumps” in display data. “The effective update rate could be considered nominally as the rate of the slowest 




component in the pathway of interest” 20 . A minimum of 15Hz is recommended for commercial Head-Up Displays 21 but 
other works show that this may be very optimistic. 22 (Note that a 15 Hz update will cause a minimum of 100 msec 
system latency due to one-half the update rate for a sampling delay of the input and 67 msec computational delay if only 
one computation cycle is necessary to compute and display the system output from the input). 
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Figure 2: Pure Time Delay Laplacian Description and Example Time History Response 

A prominent source of latency, which is not a “pure time delay,” is caused by dynamic elements in the computational 
path. A prime example would be the filtering of a head-tracker signal. While the filters attenuate noise in the head- 
tracker signal, the undesirable side effect is the addition of phase lag, which manifests itself as latency. 

In many studies, these different latency contributions, definitions, or measurements have not been well-understood. 
Actual measurements have often not been made. From the technologies of flight simulation and flight control, two 
latency measures have been developed and are accepted as standards because they are reliable, sensitive, measurable, 
and valid as they are inclusive of all delay sources and types: “equivalent time delay” and “effective time delay.” 6 

2.2 Equivalent Time Delay 

— T S 

Equivalent time delay ( e e ) is measured by comparing the frequency response of the system against a desired system 
output. Phase lag greater than the “desired” system is considered to be the “equivalent time delay” (Figure 3). For the 
analysis of aircraft control laws, this analysis sometimes gets confounded 23 by the definition of a “low order” desired 
system response. However, in the analysis of HMD systems, the desired system response is trivial; the desired 
amplitude ratio is unity (0 dB) and the phase lag is zero, as the HMD should respond without delay or amplification to 
the system input. In this case, the “equivalent time delay” can be measured directly as the phase lag of the system 
response to a sinusoidal input. While not called “equivalent delay,” researchers have generally recognized this latency 
measurement issue and in many cases, are already measuring the system equivalent time delay. 24,25,26,27 

2.3 Effective Time Delay 

Effective time delay ( e ) provides, in most situations, the analogy (if not identical numerical value) to “equivalent 
time delay,” but in the time -domain. Effective time delay is computed from a time history of the system output to a 
sharp, abrupt system input, such as a step input. The effective delay is calculated from the time difference between the 
system input and the maximum slope intercept of the system output. An example is shown in Figure 3 where the system 
input is pilot’s lateral control input and the output is the aircraft roll rate response (p). 

3. LATENCY RESEARCH IN S/EVS-RELATED APPLICATIONS 

Latency research is reviewed from the technical disciplines of flight control, flight simulation and virtual reality research 
to develop requirements for head-worn applications of S/EVS. Several obstacles with this approach are that: 1) 
inconsistent latency definitions have been used in the different research projects; 2) latency measurements have not 
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always been made; and, 3) most comparisons/experiments were not to an “established” baseline; but rather, were relative 
comparisons. Future research will, hopefully, address these deficiencies. 




Figure 3: Equivalent and Effective Time Delay Computation Examples 

From this review, conflicting evidence can be found on the effects of display delay. However, if interpreted from a 
consistent pilot-vehicle interface standpoint, the data collectively unfolds into a plausible requirements basis. 

3.1 Acceptable Latency Values: Pilot- Vehicle Dynamics System Analysis 

The engineering analysis/analytical framework from which to evaluate these data, technologies and their resulting 
influence on the pilot is provided by the pilot- vehicle dynamic system, 28 is shown in Figure 4. This diagram identifies 
the influencing factors and characteristics that “govern the ease and precision with which a pilot performs the tasks 
required in support of an aircraft role.” The key element in this dynamic system is the pilot. The pilot acts as the 
controller of this closed-loop system to complete a given task by making decisions, determining and taking corrective 
actions, and evaluating and re-evaluating the results of these actions. 



Figure 4: Pilot-Vehicle Dynamic System 

The pilot in this system responds depending upon the sources and types of information or cues received, tailored by 
individual training, experience, expertise, expectation, and real or apparent stress/workload. The information consists of 


aural, visual, and motion cues, tactile feedback from cockpit interfaces/systems such as control stick movements, and 
external disturbances, which are not necessarily correlated to the pilot’s control actions. 

Review of the research shows that the effects of HMD delay, specific to S/EVS applications, depend predominantly 
upon: 1) the required task; 2) the available visual cues; and, 3) the available motion cues, as described in the following. 

3.2 Acceptable Latency Influences: Task Dependency 

In the development of flight control systems, the United States Air Force military specification, based on years of 
research, development, test and evaluation, stipulate a maximum of 100 msec equivalent time delay to ensure that the 
aircraft flying qualities are satisfactory without improvement. Yet, with the addition of just another 140 msec (i.e., a 
total equivalent delay of over 240 msec), aircraft controllability comes into question. The key factor in the background 
of these requirements is that the task is a precisely defined, time -critical operation. The pilot must generate control 
inputs, which achieve and maintain a high level of closed-loop pilot-vehicle performance. 

When tasks require less precision or time-criticality, the pilot can relax their control inputs since a high level of system 
performance is not required. Consequently, latency effects are significantly reduced. Equivalent delays over 350 msec 
may be tolerated, 6 unlike the data shown above. This same dependence of time delay effects on task requirements has 
been found in other applications 14 as it naturally follows that the required task drives the closed-loop system and the 
criticality of the system latency. 

3.3 Acceptable Latency Influences: Visual Cue Dependency 

Acceptable latency values are shown to depend upon: the fidelity of the HMD visual cues and whether outside visual 
cues are also available and used in the accomplishment of the task. The HMD visual cues are predominately the field - 
of-view, resolution, and transmissivity of the HMD. Other factors will also be important such as color, brightness, etc. 
but data guiding these effects are not as prevalent. 

With an opaque HMD, subtending a moderate visual field-of-view (FOV), (48° H x 36° V FOV and approximately 4.5 
arc-min resolution), studies have shown that: 

• Users are very sensitive to display latency effects when they provide the only visual cues. Psychophysical 
studies 53 indicated that subjects can discriminate latency differences at least as low as 16.7 msec. 

• Virtual environment stability, or oscillopsia, which refers to the perception that the visual world appears to 
swim about or oscillate, 12 are a first-order effect of latency in opaque HMDs. As latency increased or head 
rates increased, image stability was shown to degrade. Beyond oscillopsia, a primary concern has been the 
possibility of visual- vestibular conflict, possibly leading to motion or simulator sickness 33 (discussed later). 

• Lower latency promotes the sense of virtual environment “presence”: Participants in the lowest latency 

condition (50 msec) had a higher self-reported sense of presence than a higher latency condition (90 msec) 15 to 
the extent that physiological measures of stress were statistically significant in the lower latency conditions. 

Latency of visual information is not as noticeable and detrimental to the task if the (delayed) information extends only 
into the pilots' foveal vision. 

• Ground simulation research (without motion) on latency effects in primary flight displays showed task 
performance and pilot workload degradation, although shallow, was observable with increasing display delay. 
These trends were supported by increased stick activity and degraded tracking performance. 29 The United 
States Navy established 150 msec as the maximum allowable display latency of all basic flight information. 

• Flight testing of a Head-Up Display (HUD) with delays up to 250 msec did not significantly affect task 
performance and pilot workload if the aircraft otherwise exhibited good flying qualities (little motion delay) 22 . 

A lack of prominent visual display latency effects stems from the small visual extent of the displays. Human perception 
of motion 30 is derived predominately from vestibular senses and peripheral vision cues. 31 Vestibular motion cueing is 
the most direct and accurate cue for “high frequency” motion effects, with peripheral visual cueing providing 
supplement motion cueing (albeit delayed and degraded in comparison to vestibular sensing) and spatial orientation. 
Foveal visual cues provide predominant attitude information but are relatively poor rate of motion sensors. A limited 



field-of-view HUD or primary flight display provides status information in the foveal area which are important to 
“higher-level” cognitive task decisions, but they do not provide the more powerful peripheral cues which are a 
significant component of a human's orientation cues (similar to vestibular and otolith functions) that the pilot needs for 
aircraft control. Thus, latency in small foveal displays does not impact the high frequency vestibular and peripheral cues 
that the pilot will predominately uses for motion control. HMD studies 32 have substantiated these findings in that a wide 
field-of-view display provides more stimulation and results in a more compelling display of motion. 

3.4 Acceptable Latency Influences: Motion Cue Dependency 

The role of combined motion and visual cues has been the focus of numerous studies in flight simulation. These works 
have culminated in many reference materials, including an FAA Advisory Circular (No. 120-40B) for “Airplane 
Simulator Qualification”, that stipulates that the simulator “response to abrupt pitch, roll, and yaw inputs at the pilot’s 
position be within 150 milliseconds of the time, but not before the time, when the airplane would response under the 
same conditions ” (emphasis added). This defines that the combination of math model and motion and visual system 
delays must be no more than 150 msec more than the actual aircraft response. 

The latency requirements are for those simulators certified for the highest levels of training fidelity (i.e., Level C and D 
simulators) whereas, for lesser certified levels (for Levels A and B), the response must only be within 300 milliseconds. 
Lesser certification levels denote that the training value and simulation fidelity are lower, indicative that latency greater 
than 150 msec added to the aircraft response degrades training and may introduce negative transfer of training. 

In the absence of motion cueing, latency added to a simulation tends to be less prominent of an effect on the pilot - 
vehicle system. If very little delay is present in the system, performance and the pilot’s perception of the system 
response is not as good without motion (vestibular) cueing as it is with full (vestibular) motion. Conversely, if 
significant amounts of latency are present in the system, performance and the pilot’s perception of the system response is 
not as bad without motion as it is with full motion. 7 The key factor is, again, that latency critically impacts vestibular 
cueing (i.e., actual motion cues), primarily, and through peripheral visual cues secondarily. 

3.5 Acceptable Latency Influences: Motion/Simulator Sickness 

As HMD subtend significant FOV, tending toward the generation of a vection response (i.e., visually induced perception 
of self-motion), the potential for visual-vestibular conflict grows. Studies 33 verify that a wide field-of-view HMD 
provides more stimulation and results in a more compelling display of motion. 

Visual-vestibular conflicts may degrade or cause maladaption of the vestibulo -ocular reflex or the opto-kinetic reflex. 
The vestibulo -ocular reflex generates compensatory eye movements based on vestibular senses to keep the gaze stable in 
space. The opto-kinetic reflex signals head motion and generates compensatory eye movements as an image of the 
world moves across the retina. Vestibulo -ocular reflex is dominant at high frequencies; opto-kinetic reflex dominates at 
low frequencies and modest head motion velocities. 12 

While visual-vestibular asynchronization or cue conflict has been found to be a contributor to simulator sickness, this is 
not the only factor. Other visual cue factors may be as or more critical, for instance, display resolution, scene content, 
depth perception, user background, etc. The data, in no -motion testing conditions, does show that: 

• In a driving simulator, virtual environment “presence” increased as the field-of-view increased from 60° to 
180°. 34 In correlation with increased presence, user reported increased simulator sickness, as measured by the 
Simulator Sickness Questionnaire (SSQ). (Latency was not varied.) 

• With a conformal HMD, albeit with a relatively small FOV (6.8 arc-min/pixel resolution, binocular, with 25° H 
x 19° V,100% overlap), simulator sickness ratings occurred as a result of exposure to the virtual interface, 
independent of the HMD latency. 35 Simulator sickness ratings (“negligible symptoms”) corresponded to virtual 
environment simulation only - the minimum delay condition of 48 msec, 175 (125 msec added) and 300 msec 
(250 msec added) delay conditions did not influence the SSQ ratings. The participants were seated in a chair, 
performing a visual search task, mostly in yaw. 



• With a 1280x1024 pixel resolution, binocular 60° H x 40° V FOV 36 HMD consisting of a reported nominal 
latency of 46 msec, simulator sickness ratings varied directly with time on task, not with HMD latency. The 
results lend credence to the notion that time delay per se is not sufficient for the onset of simulator sickness. 

In contrast to no-motion testing conditions, full motion testing conditions showed subtly different trends: 

• In an in-flight evaluation, 37 an HMD degraded helicopter handling qualities because of reduced visual acuity, 
limited FOV, and latency and caused pilot fatigue due to excessive helmet inertia. The HMD was provided a 
substantial FOV (105° H x 45° V FOV with 25° Binocular Overlap); but it was coupled to an NTSC sensor 
platform yielding only -11 arc min line per pixel pair resolution. As a result, the HMD acuity was one of the 
main factors affecting performance, reportedly providing only -20/120 Snellen visual acuity. Some motion 
sickness complaints (eye strain, uneasiness and helmet discomfort, stomach awareness) were reported and 
significant changes in head-movement were noted when using the HMD. Pilots reported that they reduced head 
rate movement and scanned the display with their eyes instead. The system latency was reported to be 90-120 
msec but latency was not experimentally varied. 

• In a follow-on study, 38 using the same HMD, 50 msec was reported as an effective time delay 6 for the baseline 
system latency and latency up to a total of 350 msec was added experimentally to the HMD system. In 
addition, control system delays were experimentally varied. HMD acuity was still a main factor affecting 
performance (i.e., visual acuity of -20/120). However, the effects of visual (HMD) and control system latency 
were reported to be very similar. The absence of handling qualities problems with this amount of flight control 
system delay suggests that demanding piloting tasks could not flown because of the lack of HMD visual acuity. 
Pilots were reported to adapt to the delay conditions. With increased HMD latency, the pilots tended to reduce 
their head movements, especially in yaw. Pilots reported that large yaw head movements increased 
disorientation and workload. Long visual delays, in tasks which required head -movements, prompted post- 
flight discomfort. A follow-on simulator study showed a significant increase in reported simulator (motion) 
sickness as the visual delay increased from the baseline condition to 350 msec total HMD latency. 

• In a related research area, 39 subjects performed driving tasks in a military armored vehicle using four display 
conditions: direct, periscope, head-slaved with normal camera lenses, and with wide-angle lenses. (Non- 
conformal situation is ignored for this analysis). The driver’s HMD was coupled to a 2 DOF head-slave 
stereoscopic camera system (heading and pitch). The opaque HMD provided 640x480 pixel resolution over a 
48° H x 36° V yielding approximately 4.5 arc-min resolution. In comparison of performance, driving with the 
head-slaved conformal HMD was 50% better than the current equipage (periscope) but was still only 50% as 
good as direct viewing. Three of seven subjects reported motion sickness symptoms after driving in an HMD 
viewing condition and subsequently, withdrew from the experiment. Motion sickness was not expected with 
the HMD - no reported problems in previous studies that involved direct view cameras and monitors. In this 
experiment, it was hypothesized that four factors contributed to the motion sickness occurrences: 1) Omission 
of image roll; 2) inherent delays between visual and vestibular (due to the HMD system latencies); 3) low 
image quality; and/or, 4) offset between camera viewpoint and physical position of driver. 

While not conclusive, these data indicate that visual-vestibular conflict will be a significant concern for head-worn 
applications for S/E VS. 

4. LATENCY REQUIREMENTS IN HEAD-WORN DISPLAY S/EVS APPLICATIONS 

The following latency requirements are given based on the following assumptions of what the HMD requirements will 
be for S/EVS applications: 

• The HMD will be used as a large field-of-regard display to create or augment the environment in which the 
aircraft is being operated; it is not primarily a pointing or aiming device. The S/EVS information will be 
conformally displayed. 

• The HMD will be see-through (>70% photopic transmissivity 40 ). The pilot will be able to see the cockpit 
instrument displays (unless a “see-through” mode is selected by the pilot; whereby the pilot will be able to 
display the S/EVS imagery at all aspects and thus, “look-through” the aircraft fuselage.) 



• In unrestricted day visibility conditions, the S/EVS imagery will be removed from the display (i.e., decluttered) 
and only necessary flight symbology will be provided. 

• In marginal VMC, VMC on-top, night, or IMC conditions, the S/EVS imagery will be displayed and the HMD 
provides the predominant but not only visual cues to the pilot. The degree that the S/EVS imagery will 
predominant the pilot’s visual cues will depend upon the outside (i.e., background) illumination, coloration, and 
visibility and the HMD field-of-view. 

• HMD display resolution will tend toward “eye limiting” resolutions (i.e., approaching 1 arc -min resolution). 

4.1 Measurement 

Both equivalent or effective time delay measures should be used in the analysis of HMD latencies to ensure reliability 
and repeatability of analyses. Measurement, in general, is always needed because: a) these data are the only way to 
verify performance against specification 24 ; b) these data are needed to evaluate where latency improvements can be 
made 26 ; and, c) assumed latencies don’t necessarily meet reality due to unforeseen, because, without periodic 
measurement, system latency can grow unexpectedly as a side effect from other system “upgrades.” 41 

Both equivalent or effective time delay measures should be used as numerical differences can be revealing in the 
analysis and understanding of latency effects. In general, the two measures yield the same numeric result for linear 
systems. Differences will arise for nonlinear systems or systems which are frequency -tailored, such as those that attempt 
latency “compensation” or “prediction” as discussed later. As such, both measures should be used and also, the 
equivalent time delay measurement should be computed using a fairly large frequency band. 

The methods by which to measure equivalent or effective time delay should be repeatable, easily administered, and 
preferable “in situ”; hence, special test software should be refrained from since often the special software may be 
unrepresentative of the application. Some methods for measuring video system and HMD latencies have been 
developed 24,25,27,42,43 and may be applicable. Automatic testing methods are also very desirable. 

An easily administered in-situ latency measurement technique such as a “windshield washer” test, should be considered 
as a psuedo built-in-test method (Figure 5) because of the importance that latency may have in system acceptability and 
mission success. This test involves a space-stabilized, boresight symbol and an outer “target box” of known dimensions 
located symmetrically on either side of the boresight symbol. In the presence of HMD latency, the boresight symbol 
cannot remain perfectly space-stabilized. The test requires that the user smoothly oscillate their head in azimuth (or 
elevation) at a rate which causes the space-stabilized symbol to touch the outer, target box. The head-movement rate 
data divided by the size of the target box, defines the equivalent time delay (at one frequency). (Note that the latency in 
the head movement data is immaterial to this computation; only the average rate is needed for the equivalent delay 
calculation.) While automatic methods to make these measurements are desired for repeatability of results, a user can be 
easily trained to obtain this data. Ranges of target box sizes should be used to test for linearity and multiple frequencies 
as necessary for frequency-tailoring effects analysis. 

4.2 Overall Requirement 

The end-to-end latency for an HMD supporting an S/EVS application should be no greater than 20 msec equivalent or 
effective time delay. This requirement is based on the following principles: 

• In the definition of HMD latency requirements for S/EVS, a worse-case task requirement should be assumed. 
Therefore, the latency requirements will be stringent because, as shown previously, the latency influences are 
pronounced for demanding piloting tasks. If lesser task demands are envisioned, the latency requirements may 
be relaxed, but this should not be assumed a priori. 

• HMD experiments have shown that subjects moved their heads less, but faster with large FOVs than with small 
FOVs. 44 Large FOVs are assumed to be required for S/EVS applications. Thus, latency will tend to trigger 
oscillopsia. Visual- vestibular conflicts will be a prominent concern for S/EVS applications of HMDs. 

• Others have conjectured HMD latency requirements to be: 45 50 msec preferred, 100 msec marginal, 150 msec 
unacceptable. However, the visual acuity of the HMD in this test did not approach “eye limiting resolutions”; 
hence, lower pilot-task demands were evident and the evaluation pilots noted a tendency to modify their head 



movements because of the latency. For commercially-viable S/EVS applications, the use of an HMD must not 
affect normal piloting operations; i.e., it must be non-intrusive. 
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Figure 5: In-Situ HMD Latency Test Example 


• If the S/EVS application requires accuracies of the same order of magnitude that are currently demanded from 
today’s HUDs, the latency requirements become quite demanding. From SAE ARP 5288, HUD boresight 
requirements are stipulated to minimized display errors to be consistent with the intended function of the HUD. 
The allowable display error for a conformal HUD as measured from the HUD eye reference point is less than 5 
mrad (0.286 deg). With this requirement, system latency less than 25 msec would be required for very slow 
head-rates (i.e., 10 deg/sec head-rate equates to 0.286 deg/10 deg/sec ~ 25 msec latency). Head movements of 
more than 100 deg/sec would require less than 2.5 msec system latency to remain within the allowable HUD 
error levels. Head rates of 100 deg/sec should, by no means, be considered excessive in commercial transport 
operations. 

HMD system architectures can be designed for minimum latency, which will approach this latency requirement, but to 
meet this requirement will likely require video update rates greater than the present standard of 60 Hz. 

4.3 Additional Dynamics Requirements 

Perceptual boundaries (Figure 6) from the flight control community should be adopted (until better definitions are 
available) to define acceptable “added dynamics” in HMD systems. In HMD systems, various types of prediction have 
been tried to compensate for system latency, which degrades HMD system performance. Many examples of types and 
methods are available. 46, 47 ,48, 49, 50, 51, 52 While some success has been achievable, prediction methods add “undesirable” 
dynamics or side effects. In discrimination experiments, 53 subjects became increasing correct in discriminating paired 
comparison as delay-compensation was applied, presumably because the subjects could discriminate the presence of 
predication compensation from the baseline configuration. 

These boundaries or others like them have been used for simulation cueing validation as well. 54 These added dynamics 
requirements address latency and any prediction or other dynamic system elements, which might affect the HMD system 
usability. If the added dynamics from prediction fall outside these boundaries, the method will not be “unnoticeable” 
and artifacts will be apparent. Research should be conducted to define better boundary definitions for HMDs. 


5. CONCLUDING REMARKS 


Head- worn or helmet-mounted display systems are being proposed as one method in which to create spatially-integrated, 
large field-of-regard information display systems, specifically using S/EVS. System delays or latencies inherent to these 
displays have been shown to critically influence the display utility, usability, and acceptability. Research results from 
three different, yet similar technical areas - flight control, flight simulation, and virtual reality - have been collated to 
create a proposed requirements. Consistent definitions and measurement techniques are proposed herein for universal 
application and latency requirements for Head-Worn Display S/EVS applications. Based upon the most stringent 
requirements for HMD applications of S/EVS (i.e., demanding tasks using a high resolution, large field-of-view head- 
worn display), the system latency must be less than 20 msec. Future research areas are defined. 
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Figure 6: Maximum “Unnoticeable” Differences in Gain and Phase Response. 
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